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® Virtual execution of programs on a multiprocessor system. 



@ A multiprocessor system automatically respond- 
ing to a request for executmg a new program to 
establish an extended process that spans a plurality 
of processors (101-106) each having resoun:es re- 
quired for the execution of the new program. Initially. 

3 the extended process comprises an user process 
(109) that is requesting the execution of the new 
ll^program. Stub processes (110,111) are created as 
CO required to gain access to the object code file of the 
Wnew program, to allocate a processor to execute the 
^new program, and to initilize the allocated processor 
^for execution for the new program. 
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VIRTUAL EXECUTION OF PROGRAMS 



Cross R ference to R lated Applications 

Concurrently filed herewith and assigned to the 
same assignees as this application are: 5 
TP. Bishop, et a!.. Case 4-2-1-3-2. "Inter-Processor 
Communication Protocol" . Serial No. 
T.P. Bishop, et al., Case 7-3-4^. "Controlled Dy- 
namic Load Balancing for a Muitiprbcessor Sys- 
tern". Serial No. : io 



Technical Reld 

Our intention relates to computer operating rs 
systems and more particularly to initial execution of 
a program on a multiprocessor system by an ex- 
tended process that is active on a plurality of 
processors simultaneously. 



Problem 

The use of operating systems to control the 
execution of programs on a single computer or 25 
processor and the utilization of system resources 
such as I/O devices and memory by those pro- 
grams while executing is well known in the art. One 
such operating system is the UNIX operating sys- 
tem which is described in the article by K. Thomp- 30 
son, "UNIX Implementation" the Bell System Tech- 
nical Journal, July-August. 1978, Volume 57. Num- 
ber 6. The UNIX operating system described in this 
article is designed to control the resources of a 
single processor. The execution of a new program as 
is described on page 1933 of this article which 
describes the standard "fork" and "exec" system 
calls. The procedure is to execute the fork system 
call which replicates the executing process into a 
child and parent process. These processes share 40 
the same program, but have different data storage. 
The child then executes the exec system call. The 
execution of the exec system call results in a new 
program being executed. Further information on the 
Unix operating system is given in the book of M, J. 45 
Bach entitled The Design Of The Unix Operating 
System . Prentice-Hall. 1986. Englewood Cliffs, New 
Jersey. Other known operating systems have simi- 
lar system calls. 

Whereas the exec system call of the UNIX so 
operating system described in the article by 
Thompson is an extremely effective mechanism for 
causing the execution of a new program, it is not 
capable of operating in a multiprocessor environ- 
ment where the exec system call executes on one 



ON A MULTIPROCESSOR SYSTEM 

processor but causes th new program to com- 
mence running on yet a second and unspecifi d 
processor. O^raUng systems for causing the ex- 
ecution of specialized procedures or subroutines 
on other processors in a multiprocessor nyiron- 
ment are known. However, these systems require 
that the procedure or program already exist on the 
other processor. One such systeim is described in 
U.S. Patent 4.530.051, of J. W. Johnson et al. This 
patent describes a multiprocessor system wher by 
one processor can cause the execution of a proce- 
dure, or as It is often called a subroutine, n 
another processor. The problem is that the proce- 
dure must already exist on the other processor 
before execution. A similar system is described in 
the article by P. Jackson, "UNIX Variant Opens a 
Path to Managing Multiprocessor Systems". Elec* 
tronics . July 28. 1983. In this article, a system is 
described whereby a program executing on one 
processor can obta'n access to an I/O devic . such 
as a disk drive, by causing a procedure to run on 
another processor which controls the disk drive. 
When Information is obtained from the disk drive 
the other processor transfers this to the requesting 
processor. This article does not describe the gen- 
eral capability of automatically causing a program 
to be retrieved from a disk and executed on the 
remote systert) without the program having b n 
previously designated as existing on that system. 

Another method of executing programs in a 
second processor from a first processor is dietailed 
in the "Remote Procedure Calf Protocol Specifica- 
tion" Manual. Part No. 800-1177-01, Sun Micro- 
systems, inc.. 2250 Garcia Avenue. Mountain View. 
California. 94043. This manual gives details on the 
remote procedure call as implemented by Sun 
Microsystems, which is an addition to the UNIX 
operating system. The remote procedure call al- . 
lows procedures to be executed on a second pro- 
cessor from a first processor. 

The problem that exists in the prior art fs that 
there is not a general way to cause the execution 
of a new program in - a multiprocessor system so 
that the system call executing that program is free 
of considerations dealing with the loading of the 
various processors within the multiprocessor sys- 
tem, the physical location of the program wh ther it 
be in memory or in file system, and the setting up 
of the necessary signaling paths if the execution of 
the program requires the system call performing 
this function to span a number of processors within 
the multiprocessor system. 
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Solution 

This invention is directed to solving these and 
other problems and disadvantages of the prior art. 
According to the invention, a multiprocessor sys- 
tem is automatically responsive to a request for 
executing a new program thus establishing an ex- 
tended process that spans a plurality of processors 
each having resources required for the execution of 
the new program, initially, the extended process 
comprises a primary process that is requesting the 
execution of the now program. Advantageously, 
auxiliary processes are created as part of the ex- 
tended process as required to gain access to the 
object code file of the new program, to allocate a 
processor to execute the new program, and to 
initialize the allocated processor for execution of 
the new program. 

Advantageously, a method for starting the ex- 
ecution of a new program in a multiprocessor sys- 
tem comprises the steps of: determining a proces- 
sor associated with the object code file of the new 
program, creating a first auxiliary process on the 
determined file processor upon the file processor 
having been determined to be different than the 
initiating processor, allocating another processor for 
executing the object code file of the new program, 
creating a second auxiliary process on the allo- 
cated executing processor upon tiie executing pro- 
cessor being different tiian the initiating processor 
or file processor, transferring process information 
from tiie first primary process on the initiating 
process to the second auxiliary process, transform- 
ing the second auxiliary process into tiie primary 
process, and executing tiie object code file by the 
primary process In conjunction with the first auxil- 
iary process. 

Advantageously, the allocating step comprises 
the steps of communicting a first packet from the 
initiating processor to the file processor to request 
ttiat the first auxiliary process read a portion of the 
object code file, reading the requested portion of 
the object code file by the first auxiliary process, 
and communicating a second packet containing the 
read portion to the initiating processor from the file 
processor. ^ 

In addition, the system has a designated host 
processor which executes a process manager func- 
tion that performs processor assignment and the 
allocating step further comprises the steps of: com- 
municating a third packet to the host process or to 
request that the processor assignment be made for 
the new program, assigning the executing proces- 
sor to execute the new program by the host pro- 
cessor running the process manager function, and 
communicating tfie assignment information via a 
fourth packet to the initiating processor. 

Also, a portion of the object code file contains 



a first set of processor assignment parameters and 
the initiating proc ssor has a second set of proces- 
sor assignment parameters stor d in a subset of 
th second auxiliary process Information and the 
5 third packet contains tiie first and second sets of 
processor assignment parameters and the assign- 
ing stop further comprises the steps of reading tiie 
sets of processor assignments parameters from the 
third packet and designating the executing proces- 

10 sor for the processor assignment in response to 
the sets of processor assignment parameters. 

In addition, the process information associated 
with ttie first primary process allows tiie first pri- 
mary process to function as the primary process of 

75 the extended process and the transf^ning step 
comprises the steps of reading the process in- 
formation by the initiating processor, forming the 
process information into a fifth packet by the initiat- 
ing process, and transmftting the fifth packet from 

20 the initiating processor to tiie executing processor. 

Advantageously, the h-ansforming step com- 
prises the steps of reading tiie process information 
from the fifth packet and storing the process in- 
formation into tiie second auxiliary process and 

2S converting tiie second auxiliary process into the 
first primary process. Also the step of converting 
changes ttie original first primary process into a 
third auxiliary process. In addition, tiie step of 
convertin further comprises the step of sending a 

30 sixth and seventh packet to the first and third 
auxiliary processes, respectively, to inform the lat- 
ter processes that the second auxiliary proc ss has 
become the primary process and that the fonm r 
primary process is now tiie tiiird auxiliary process. 

35 These and otiier advantages and features of 
the present invention will become apparent from 
the following description of an illustrative embodi- 
ment of tiie invention taken togetiier witii tiie draw- 
ing. 

40 

Brief Description of the Drawing 

FIG. 1 illustrates, in block diagram form, a 
45 multiprocessor system for utilizing the present in- 
vention; 

FIG. 2 illustrates, in flowchart fomri. tiie func- 
tions performed during the execution of an ex c 
system call by the multiprocessor system of FIG. 
so 1; 

FIG. 3 illustrates, in block diagram form, the 
interconnection of an extended process for a sul3- 
set of the processors of FIG. l ; 

FIG. 4 Illustrates, in greater detail, the soft- 
55 ware interconnection of FIG. 3; 

FIG. 5 illustrates, in block diagram form, the 
file control structure for an extended process ex- 
ecuting on the processors of FIG. 1 : and 
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FIG. 6 illustrates, in bfock diagram form, the 
fil control structure for accessing aout files for an 
extended process executing on the processors of 
FI3. 1. 



Detailed Description 

FIG. 1 shows a multiprocessor system having a 
plurality of computers 101 through 106 intercon- 
nected by bus 107. Some of the computers illus- 
trated in HG. 1 have particular functions. For exam- 
ple, computer 101 is considered to be the host 
computer, and computers 105 through 106 may be 
designated as computationai servers of file servers. 
Each computer operates under control of an op- 
erating system kernel which Illustratively is a ver- 
sion of the UNIX operating system described in the 
article by Thompson. Whereas the operating sys- 
tem kernel described in Thompson is restricted to 
only a single computer, the kernels of FIG. 1 allow 
a process to be extended over a number of com- 
puters. This extended process is a collection of 
individual special processes running on separate 
computers and is described in greater detail later 
in this section. These special processes are also 
referred to as primary or user and auxiliary or stub 
processes. Each kernel associated with the ex- 
tended process maintains a distinct process neces- 
sary to allow the extended process to function on 
the computer controlled by the associated kernel. 
Each computer has associated memory and VO 
devices; however, certain computers may be inter- 
connected to special I/O devices such as tele- 
communication data interfaces or mass storage de- 
vices. 

The initiation of a new program in the system 
illustrated in FIG. 1 results in that program being 
automatically assigned to an unspecified computer 
that has processing capacity to spare or which has 
special I/O resources required by the program, The 
unspecified computer may be the same computer 
executing the request or a different computer. The 
execution of the program can be distributed over a 
number of computers utilizing one computer which 
has processing capacity and yet using one or more 
computers which have the necesary files or special 
I/O capabilities. When program execution is distrib- 
uted, an extended process is created. The opera- 
tion of initiating the execution of a program so as to 
allow the execution of the program to be performed 
among a plurality of computers and yet making this 
operation transparent to the application program- 
mer is ti^e subject of this invention. 

The allocation of resources and dynamic load 
balancing is performed by process manager (PM) 
function 108 being executed by computer 101 
which is designated as the host computer of the 



system of FIG. 1. The initiation of the execution of 
a new program is performed by the exec system 
call that is modified by this invention so as to allow 
the operation on a multiprocessor syst m. Consider 
5 the following example which illustrates the execu- 
tion of th xec system call. Olduser process 109. 
on computer 102. executes the exec system call. 
The end result is that the new program is ven- 
tually executed by newuser process 111 on com- 
w puter 104. Initially, the file containing the new pro- 
gram is in the file system of computer 103 and is 
accessed by a.out process 110. Computers 105 
and 106 also have resources that will be utilized by 
newuser process 111. 
75 Jn the previously referenced article by Thomp- 
son, it was noted that a process is a software unit 
which requires text, data and bss areas of memory 
and which is identified to the operating system by 
a process control block. In the operating syst m 
20 described by Thompson, the process coritrol block 
is contained in one area of memory since tiiat 
operating system is executed on a uniprocessor 
system. In the system illustrated in FIG. 1. the 
process control block is distributed among all th 
25 computers which are associated with the extended 
process. The extended process comprises pro- 
cesses 112. 110, 111 and possibly processes lo- 
cated in computer 105 and 106 after the xec 
system call is finished* The extended process con- 
so sists of a user process and a number of stut> 
processes. The user process has text, data and 
bss areas of memory in addition to the process 
control block. A stub process contains only a por- 
tion of the process control block relating to op rat- 
35 ing system functions pertaining to that particular 
computer's operations witii respect to the extended 
process as required at any point in time. 

Upon olduser process 109 executing th exec 
system call, the kernel of computer 102 transmits a 
40 packet to computer 103 to obtain the head r por- 
tion of the a.out file via a.out process 110 so as to 
determine the type of resources required to ex- 
ecute this program. The kernel of computer 102 
then transmits a packet to the kernel of computer 
45 101 requesting allocation of resources for the ex- 
ecution of newuser process 111. In response to this 
request, the kernel of computer 102 executes pro- 
cess manager function 108 In our present example, 
the kernel of computer 102 transmits back a mes- 
50 sage designating that computer 104 is to execute 
newuser process 1 1 1 after executing process man- 
ager function 108. Further information concerning 
the operations of process manager function 108 is 
illustrated in the copending application of Bishop et 
55 al.. Case 7-3-4-4. The kernel of computer 102 then 
transmits process control information to the kernel 
of computer 104 so that the latter kernel can setup 
newuser process 111 and stub processes in com- 
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puters 102, 105. and 106 for the future execution of 
the extended process. 

Once this initialization has be n performed, the 
kernel of computer 102 passes the execution of the 
ex c system call to the kernel of computer 104. 
The latter kernel obtains the a.out file from com- 
puter 103. The kernel of computer 104 also trans- 
mits messages to the kernels of the other comput- 
ers informing them that the user process which 
was initially oiduser process 109 has migrated to 
computer 104 and is now newuser process 111. 
Oiduser process 109 new becomes a stub process. 
The kernels of the other computers will now direct 
any signals for oiduser process 109 to newuser 
process 111. Further, the kernel of computer 104 
transmits a message to the kemel of computer 102 
to recover all signals transmitted to oiduser pro- 
cess 109 that arrived at computer 102 before the 
other computers were informed that the extended 
process had migrated to computer 104. Once 
newuser process 11 1 has been set up and begins 
to execute, it can utilize the resources of the other 
computers as required via stub processes that 
were created in these computers. If. during the 
execution of the program, it is necessary to access 
a computer that was not initally designated as 
being part of the extended process, then the op- 
erating system of computer 104 requests the cre- 
ation of a stub process on that computer necessary 
to continue execution of the program. 

FIG. 2 illustrates in greater detail the execution 
of the exec system call and creation of the ex- 
tended process for the present example. Upon 
execution of the exec system call by oiduser pro- 
cess 109. decision block 202 is perfomned. The 
exec system call may specify parameters for in- 
fluencing the processor assignment. Decision block 
202 determines whether or not the file containing 
the a.out file is iocal to computer 102 or is on a 
remote computer. Since the file Is on computer 103 
in the present example, it is remote: and if a stub 
process does not already exist on computer 103 
for the present extended process, a packet is sent 
to create a stub process on computer 1 03. In 
response to the packet, the kemel of computer 103 
creates a.out process 110 that allows access to the 
a.out file. a.but process 110 then becomes part of 
the extended process. Block 206 accesses the 
a.out file located on computer 103 via a.out pro- 
cess 110. The header information is read from the 
a.out file and is stored in the process control block 
of a.out process 110. The kernel of computer 103 
then transmits a subset of the header to computer 
I02's kernel which stores the subset in the process 
control block of oiduser process 109 in computer 
102. The information obtained from the a.out file at 
this point specifies the size of the a.out file and 
may specify parameters for influencing the proces- 



sor assignment decision. Aft r obtaining the in- 
formation from the a.out fil , the kernel of computer 
102 transmits a packet to the kernel of comput r 
101 requesting that th kernel execute process 

5 manager function 108 to select a computer upon 
which newuser process 111 is to assigned at block 
208. This packet contains the information obtained 
from the a.out file in block 206 and any parameters 
regarding processor assignment in the exec sys- 

10 * tern call. PM process manager function 108 is 
responsive to this packet to validate an explicit 
assignment if one existed in the a.out or exec 
system call information or to perform a dynamic 
load balancing for the multiprocessor system illus- 

75 trated in RG. i in order to nriake a processor 
assignment for newuser process 111. In the 
present example, newuser process 1 11 is assigned 
to computer 104. 

Next, the kemel of computer 102 executes 

20 block 210. The execiition of block 210 results in 
the arguments of the exec system call being r ad. 
The kemel of computer 102 is responsive to these 
arguments and any environment variables from the 
oiduser process I09's address space to transf r 

25 this information into a system work area formatting 
this information into an initial stack for newuser 
process 111. Block 212 Is next executed which 
releases the resources of oiduser process 109 
back to the operating system of computer 102. In 

30 particular, the address space of oiduser process 
109 is released. 

The actions just performed represent a pre- 
execution stage of the exec system call. If th 
newuser process Is present on a different comput r 

35 than the oiduser process, then blocks 220 through 
238 are executed before bbcks 240 through 250. 
In the present exempte. the kemel of computer 102 
executes blocks 220 through 228. and the kernel of 
computer 104 executes blocks 230 through 238. 

40 However, if the oiduser and the newuser proc sses 
are on the same computer, then the blocks 240 
through 250 Illustrated in FIG. 2 are executed at 
this point in time. Decision block 214 determin s 
whether or not the newuser and oiduser processes 

46 are on different computers. In the present example, 
oiduser process 109 is on computer 102 and 
newuser process ill is on computer 104. If a stub 
process does not already exist on computer 104. 
the kernel of computer 102 executes block 220 

50 which results in a packet being transmitted over to 
the kernel of computer 104. This packet requests 
that a stub process be created which will become, 
newuser process 111 on computer 104. The kern I 
is responsive to this request to create a skeleton 

55 stub process by performing a kernel fork function 
on a prototype stub process. Each kernel of FIG. 1 
maintains a copy of the prototyp stub process for 
the purpose of creating stub processes. The kernel 
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of computer 102 then executes black 222. The 
latt r block results in the transmission of a migra- 
tion pacicet from computer 102 to computer 104. 
The packet contains the initial process control in- 
formation for newuser process 111. That informa- 
tion was foiTDatted in bJock 210.. The migration 
packet contains the information necessary to trans- 
form (he stub tor newuser process 111 on com- 
puter 104 into a viable user process of an extended 
process. ViabiRly fs defined here to mean that tfie 
newuser process has all the infomnation necessary 
to exit or terminate gracefully If required. A graceful 
exit is one where all parts of the extended process 
can be removed from all the computers of FIG. 1 if 
ft Is necessary to terminate the extended process. 

The principal information contained in th© mi- 
gration packet fs the reconnect^on data for the stub 
processes and information defining open files of 
the extended process. This data is used to reattach 
the stub processes and files that had been at- 
tached to olduser process 109 to newuser process 
111. Th© reattachment is performed by rearranging 
the virtual channels and discussed with respect to 
FIG. 3. Certain crudal data from the process con- 
trol block defining process group ID, parent pro- 
cess (0, flagword. user fO. group ID. current drrec- 
tory. private root directory, new argument pointer, 
and various timekeeping fields are afso commu- 
nicated via the migration packet. The kernel of 
computer 104 is responsive to the migration packet 
from oJduser process 109 to install the data con- 
tained in this packet in the newuser process lIVs 
control block and to issue reconnect messagos to 
all of the stub processes in the other computers. 
After bfock 232 and 234 have been perfomied, the 
newuser process 11 1 is considered viable. 

The reconnect messages transmitted by block 
234 to the other computers cause the kernels of 
these other computers to transform those comput- 
ers' portion of the process control block of the 
extended process to new point to newuser process 
111 f'n computer t04 rather than olduser process 
109 in computer 102. The significance of this re- 
connection is that any signals generated for the 
extended process by stub processes of other com- 
puters are now transmitted to newuser process 1 1 1 
rather than to olduser process 109. 

The operating system of computer 102 now 
executes block 224 which resvHs in the transfer of 
the exec arguments and other Information to 
newuser process 111 via the kemef of computer 
104 by a series of packets from the kernel of 
computer 1 02. The newuser process 111 is then 
built up by installing these packets into the 
newuser process 111 address space on computer 
704 by the k me\ of computer 104. This transforms 
the newuser process 111 Into a more complete 
user process of the extended process. 



The k©m ! of computer 104 then executes 
block 236 that sends a m ssage to computer 102 
causing the executfon of bfock 226 which results in 
olduser process 109 being turned into a stub pro- 

5 cess. The k rnel of computer 104 then transmits a 
request at block 238 to th kern I of computer 102 
for all signals destined for the user process of the 
extended process that may be stored for olduser 
stub process 109 in computer 102. Computer 1Q2*s 

fo kernel responds to this message by executing 
block 228 which transmits these signals to block 
238. 

The kernel of computer 104 now executes the 
blocks 240 through 250 in FIG. 2. These fatter 

is blocks are executed in the same manner regard* 
less of whether or not the olduser process and the 
newuser process are on the same computer. First 
the kernel accesses the a.out file located on com- 
puter 103 via aout process 110 to obtain the a.oiA 

20 header by execution of block . 240. Utilizing the 
header information, the kernel of computer 104 
builds the newuser process Ill's address space, 
including space for text. data, and bss, by loading 
the various sections from the a.out file into com- 

25 puter 104 from computer 103 by execution of block 
242. 

After performing this function, the kernel then 
executes bfock 244 so as to close any files which 
were associated with olduser process 109 but will 

30 not be associated with newuser process 111. 

The files that are to be closed are determin d 
by the application programmer. The programmer 
marks the files to be closed in a standard UNIX 
manner using the fcnti system call prior to execu- 

35 tion of the exec system call This information is 
stored in the process control block of old user 
process 109 and is fater transferred to newuser 
process 111. After closing all of the marked files, 
the kemel of computer 104 executes block 246 so 

40 as to reinitialize the an-ay of signal-handler fields 
which contain an entry defining the action to be 
taken upon receipt of a signal. Each entry can 
specify one of the following: default value, ignore 
vafue. and a pomter identifying a function that 

45 services that particular signal. The signals w re 
transmitted from computer 102 to computer 104 in 
block 228 and combined with signal entry 423 in 
block 238. The block 246 sets all entries in the 
signal array pointing to functions to the default 

so value but any entry that contains an ignore value is 
not modified. When a signal is received for a 
process, the kernel accesses the process control 
block for that process and stores the signal In the 
sig entry, such as entry 423 as illustrated in FIG. 4. 

55 When a signal is handfed by the kernel, the signal 
number is used as an index to access the signal 
array. If tiie default value is acc ssed. the process 
wiJI normally terminate. If the accessed entry con* 
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tains the ignore value, no action is taken. If the 
accessed entry contains a pointer, then the func- 
tion identified by the pointer Is executed. When the 
application runs, the signal system call will be used 
to configure the array to the requirennents of that 
program. Further informa^ory on the hantilmg of 
signals can be found in the aforementioned book ot 
Bach. Next, the kernel of computer 104 executes 
block 248 which reinitializes any memory manage- 
ment information required for newuser process 
111's new address space and completes all other 
housekeeping chores. Rnally. control is turned over 
to newuser process 111 so that the program can 
now execute at block 250» 

FIG. 3 illustrates in greater detail a portion of 
the extended process resulting from the execution 
of the exec system call. New process 111 is the 
user process of the extended process and pro- 
cesses 112 and 110 are stub processes of the 
extended process. All processes of the extended 
process share a common pid number. Virtual chan- 
nels are established between the user process of 
the extended process and the stub processes at 
the time the stub processes are established. TTiose 
channels are utilized for the communication of 
packets between the user process and the stub 
processes. Stub processes of the extended pro- 
cess do not directly communicate with each other. 
In addition, all communication from other pro- 
cesses within the system Illustrated in FIG. 1 are 
directed to the user process of the exterided pro- 
cess. Each of the computers illustrated in FIG. 1 
maintains a proc pointer table such as 301 through 
303 of FIG. 3. The pid number is utilized by the 
kernel to point into a proc pointer table such as 
tables 301 through 303 to obtain the pointer such 
as 304 through 306 to find the designated process. 
For OKampie, the pid number in computer 104 is 
used to access entry 306 from proc pointer table 

303 that points to proc table 309 via path 312. 
Similarly, the pid number utilizes to access entry 

304 of proc pointer table 301 to obtain path 310 to 
proc table 307. 

Virtual channels 313 and 314 are directly asso- 
ciated with the processes. The identification of 
these channels is established within the proc table 
of the individual processes. A user process of the 
extended process has a virtual channel to each of 
the stub processes. However, each stub process of 
the extended process, such as host process 112. 
has only one virtual channel; and that channel is to 
the user process of the extended process, such as 
newuser process 111. 

FIG. 4 illustrates in greater detail the memory 
utilized by host process 112 and newuser process 
1 1 1 and further demonstrates the differences be- 
tween a user and stub processes of the extended 
process. For each of the latter processes, portions 



of the proc table and the ubiock are illustrated. In 
addition, the port tables and the achan lists are 
shown. These latt r tabl s identify the virtual chan- 
nels between the processes. Not Illustrated for 

5 newuser process 111 are the text. data, and stack 
areas that are utilized by this process during x- 
ecution. Similarly, the stack area of memory is not 
illustrated for host process 112 which is a stub 
process of the extended process. Newuser process 

70 1 1 1 is the user process of the extended process. 

Consider now in detail the entries of proc ta- 
bles 307 and 309. TTie entries illustrated for tables 
307 and 309 are only a portion of the entri s that 
would exist in these tables. The nice entry . 401 or 

75 421 defines the scheduling priorities of a process. 
Nice entry 401 is fixed on a system-wide basis by 
the system administrator which is true for all stub 
processes. Since nice entry 421 is for a user 
process, ft is adjustable by the user executing a 

20 specific system call allowing the level of priority to 
be reduced or by actions taken by the system 
administrator or superuser which can increas or 
decrease the level of priority. Entries 402 and 422. 
pid. define the process identification number which 

35 is the sanie for both entries. The pid number is 
given on an extended process biasis so that the 
user process and af) stub processes of the ex- 
tended process have the same pid number. 

The sig entries 403 and 423 coupled with the 

30 status entries 406 and 426 are used to handle 
signaJs between processes, in addition, the status 
word of the proc tables also contains conventional 
UNIX system type infonmation. On a stub, the sig- 
nals flags contained in the status word indicate 

95 whether or not a signal has been received from th 
user process which In the present example is 
newuser process 111. Host process 112 is respon- 
sive to the receipt of a signal that is stored in entry 

403 to perform different operations depending on 
40 whether the present operation is intenrupt'ble or 

not. In newuser processor 111, the signal flags in 
status entry 426 are used to indicate wheth r a 
signal has been transmitted to host process 112. In 
addition, the signal ffags in entry 426 a/so k ep 

45 track of whether or not the message was ever sent 
to a stub process such as host process 112. Tills 
latter indication is utilized to facilitate cleanup of 
different types of operations at a latter point in 
time. The type of signal that is received is stored 

so by newuser processor 11 1 in sig entry 423. 

With respect to the parent pid (ppid) entries 

404 and d424. these entries are used to keep track 
of the identity of the parent process of the x- 
tended process. This is a conventional UNIX sys- 

55 tern type field. However, in the extended process, 
the ppid entry is only valid on the process of the 
extended process that is b ing executed by host 
computer 101 which is host process 112 in the 
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present xemple. Advantageously, this reduces the 
amount of communication of packets between var- 
ious stubs and th us r process of the extended 
process. The user proc ss of the extended process 
does not have a valid ppid entry unless the user 
process of the extended process is resident on the 
host computer. 

The bicid variable is used in the following man- 
ner to determine whether or not a particular pro- 
cess is a stub or user process of the extended 
process. Each processor of the multiprocessor sys- 
tem illustrated in FIG. 1 stores the processor's 
identification number in a variable MYLOC whose 
contents is the ident'ficafion number of the proces- 
sor. Regardless of whether it is a stub or user 
process, the bicId variable always contains the 
indentiflcation number of the processor that is ex- 
ecuting the user process of the extended process. 
In the present example, this is computer 104. Entry 
425 is identical to the contents of the MYLOC 
variable maintained by the Icemel of computer 1 04. 
Entry 405 also contains the contents of the MYLOC 
variable of computer 104 hence does not match 
the MYLOC variable of computer 101. The kernel 
of a particular computer detemines whether or not 
it Is executing the user or stub process by compar- 
ing its bicid variable with the MYLOC variable. If a 
match results, then the kernel is executing the user 
process of the extended process. 

The ubiock page entries 407 and 427 contain 
address information to setup a virtual address to 
gain access to the ubiocks which are illustrated as 
ubiocks 418 and 438 in FIG. 4. The latter ubiocks 
contain different information depending on whether 
the ubiQck Is the stub process of the extended 
process, such as host process 112. or the user 
process of the extended process, such as newuser 
process 111. Advantageously, many of the entries 
of these ubiocks are similar to those for the 
ubiocks for the UNIX system described in the 
aforementioned article by Thompson. Entries 408, 
409. and 410 contain zero for the host process 112 
since a stub process of the extended process has 
no text stack or data memory areas. When the 
kernel is performing functions associated with the 
stub process, the kernel maintains a kernel stack 
unique to that stub process in the ubiock of the 
stub process. This is similar to the manner in which 
the kernel stack is maintained in a single processor 
UNIX system when the process is executing in the 
kernel mode. Entries 428, 429 and 430 contain the 
necessary information so that the user process of 
the extended process, newuser process 111. has a 
text area, data area and a stack for the execution of 
the program that was obtained from the a.out file 
as described eariier. 

In th user process of thfe extended process, 
the variables tockip and exjecip, entries 431 and 



432. respectively, are used to identify the old arid 
new a.out tiles. In the present example, computer 
103 is storing the n w a.out , and computer 103 is 
storing the old a.out file. Entry 431 points to the old 
5 a.out file, and entry 432 points to the new a.out file 
which is executed as a result of the exec system 
call. If both files are local to the processor execut- 
ing the user process of the extended process, then 
tliese entries are pointers which point into the 
10 inode table maintained by the kernel of the local 
processor in a standard UNIX system manner to 
Identify the local files. The system file table is not 
used since the a.out files are not used by the 
process directiy but rather by the kernel. Howev r. 
15 if the file is remote. e.g. associated with another 
processor, the entry contains an identification of an 
entry in the user process port table such as port 
table 440. For example, this entry in port table 440 
then identifies the virtual channel and remote en- 
20 tries such as 411 or 412 are identified In a stub 
process of the extended process. For example, if 
entry 431 Indicates a remote file, then it points to a 
corresponding entry 41 1 In the stub process asso- 
ciated with the processor that is local to the r mote 
25 file. More information concerning these entries is 
given with respect to FIG. 6. 

In general, the ubiock of a stub process of the 
extended process contains three types of entries 
with respect to the ubiock of the user process. The 
30 first type is entry 408 which is never used in the 
stub process but is used In the normal way in the 
user process. The second type of entry in th 
ubiock of the stub process is an entry which is 
always used, and an example of this is the acptr 
3S entry 414 which is descnlDed in greater detail later 
in this section. The third type of entry in the ubiock 
of the stub process Is an entry which is only 
populated as needed with the necessary informa- 
tion being transferred over from the user process 
40 of the extended process when the requesting pack- 
et to perform a particular function is transmitted 
from the user process to the stub process. One 
example of this is the dirp entry 413, and other 
examples are entries 411 and 412. The entry 413 
45 is a pointer which points to the path name. In the 
user process of the extended process, this entry 
always points into the users address space to 
designate the path name. However, in a stub pro- 
cess, the infomnatlon concerning the path name is 
50 received from the user process and is then stored 
by the kernel at a convenient location. At this point 
and time, the kernel sets the dirp entry to point to 
the path name. An example of when the path name 
is transmitted is during an open system call. Since 
55 the open system call is executed on the processor 
executing the us r process of the extended pro- 
cess, the path name information is not available on 
the stub process. 
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The acptr entries 414 and 434 point, to the 
achan list 419 and 439. respectively. An achan list 
of a user process may have a multitude of struc- 
tur s each defining a virtual channel to a stub 
process. In the prss nt xample. newuser process 
111 *s achan list 439 has a numb r of structures 
with each defining a virtual channel to a stub pro- 
cess of the extended process. Each structure com- 
prises link and port pointers and other information 
for virtual channel operation. These structures are 
linked together by the link pointers 442 and 435 in 
a link list. The list is terminated upon the contents 
of the last link pointer being a null. The virtual 
channels are identified by the port pointers. The 
port pointers 431 and 436 point into port table 440. 
The newuser process 111 identifies the virtual 
channels asssociated with it by utilizing acptr entry 
434 to point into the link list associated with 
newuser process 111 in achan list 439. The port 
pointers then are utilized to access port table 440. 
Host process 112 similarly has acptr entry 414 
which points into the staictures containing pointers 
415 and 416 of achan list 419. Since a stub pro- 
cess of the extended process can never have more 
than one virtual channel, the link pointer 415 termi- 
nates the link list since the contents of pointer 415 
are null. Port number 417 defines virtual channel 
314 to newuser process 111. Further detail con- 
cerning the virtual channels is given in the copen- 
ding application of Bishop et al.. Case 4-2-1-3-2 
which is hereby incorporated by reference. 

Consider the virtual channels in light of blocks 
234 and 236 of FIQ. 2. When the reconnection 
information is initially transmitted by block 234. the 
stub processes of the extended process have vir- 
tual channels set up with olduser process 109 
which at that point in time is the user pn^cess in a 
similar manner as illustrated in RG. 4. Upon receiv- 
ing the reconnect information, each receiving ker- 
nel updates the port table and the other channel 
information resulting in the stub process' virtual 
channel being connected to newuser process ill 
rather than olduser process 1 09. Part of operations 
performed by block 236 is to transform achan list 
439 and port into the tables illustrated in FIG. 4. 
Block 226 changes olduser process 109's tables 
into tabies similar to those illustrated for host pro- 
cess 112 in FIG. 4. FIG. 5 illustrates the manner in 
which the files of the extended process are iden- 
tified by the user process of the extended process, 
if a file is local to the processor that is executing 
the user process of the extended process, such as 
computer 104 and newuser process 111, then the 
standard UNIX system file control structure is uti- 
lized. For example, local file 507 is identified via 
entry 504 of file table 501 . The latter table is part of 
the ubiock of newuser process 11 1 and is referred 
to as the u ofile structure. The contents of entry 



504 in turn identify entry 505 of system file table 
502. The system file table 502 is then utilized to 
point to inode table 503 and entry 506. Entry 504 is 
identified in table 501 by using the file descriptor 

5 numb r associated with fit 507 in a nonmal UNIX 
system manner. Entry 506 then identifies the local 
file 507 in a normal UNIX system manner. If the fil 
is remote such as remote file 507, which is as- 
sumed to be local to computer 105. then entry 520 

TO identifies that this file is remote and rather than 
pointing Into system file table 502 points into port 
table 440. The file descriptor number for file 517 is 
used to access entry 520. Entry 508 is identified in 
table 440 by entry 520 of port table 440 which 

75 identifies virtual channel 509. Virtual channel 506 is 
interconnected as previously described into file 
process 510 which is a stub process of the ex- 
tended process. A packet containing the file de- 
scriptor number for file 517 is transmitted to the 

20 kernel of computer 105. The latter kemel us s the 
file descriptor number to Identify entry 514 of file 
table 51 1 which Is in the ubiock of file proc as 510. 
The file control structure then identifies remote file 
517 in a nonmal UNIX manner via entries 515 and 

25 5 1 6 in tables 51 2 and 51 3, respectively. 

FIG. 6 illustrates in greater detail the utilization 
of the lockip and execip variables in th us r 
process of the extended process and the remlockip 
and remexicip variables in a stub process of the 

30 extended process. PIQ. 6 shows that the state of 
ttie multiprocessor system illustrated in RG. 1 be- 
fore the execution of block 224 of FIG. 2. At tfiis 
time, the user process of the extended process is 
olduser process 109 t>eing executed by comput r 

35 102. FIG. 6 assumes that tiie original a.out fil 
which was used to execute the exec system call, js 
local to computer 102. In the present example, the 
original a.out file is old a.out file 607. The n w a.out 
file from which the new program is to be obtained 

40 is remote and is associated with computer 103. 
This file is denoted as a.out file 617. The stub 
process of tiie extended process being executed 
on computer 103 is a.out process 110. lllustrat d 
for olduser process 109 is its ubiock 601 and port 

45 table 622. Illustrated for a.out process 110 is Its 
ubiock 611. Entry 604 of ubiock 601 of olduser 
process 109 identifies old a.out file 607 in a stan- 
dard UNIX system manner utilizing entries 604 of 
ubiock 601 and entry 606 of anode tabi 603. 

50 Since the lockip variable identifies a local file, there 
is no corresponding remlockip variable used in any 
stub process associated witii the extended pro- 
cess. 

The now a.out file, however, is remote from 
55 computer 102. Entry 620 of ubiock 601 rath r than 
pointing into inode table 603 points to entry 621 in 
port table 622. Entry 621 identifies virtual channel 
509 which is connected to ubiock 610 of .a.out 
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process 110, Entry 614 of ubiock 610 then points 
entry 616 of inode table 613 to a.out file 617. Since 
the old a.out file is local to computer 102. entry 
623, remlockip variable, is not utilized in ubiock 
611. 



Claims 

1. A method for initiating the execution of a 
new program represented in a object code file in a 
multiprocessor system having a plurality of proces* 
sors, 

CHARACTERIZED BY 

determining a file one of said processors asso- 
ciated with said object code file of said new pro* 
gram by a first primary process being executed by 
an initiating one of said processors; 
creating a first auxiliary process on said file one of 
said processors upon said file one of said proces- 
sors having been detennined to be. different from 
said initiating one of said processors; 
allocating an executing one of said processors to 
execute said object code file of said new program; 
creating a second auxiliary process on said execut- 
ing one of said processors upon said executing 
one of said processors being different from said 
initiating one of said processors; 
transferring process information from said first pri- 
mary process to said second auxiliary process: 
transforming said second auxiliary process into a 
second primary process; and 
executing said object code file, by said second 
primary process in conjunction with said first auxil- 
iary process. 

2. The method of claim 1 
CHARACTERIZED IN THAT 

said allocating step comprises the steps of: 
communicating a first packet from said initiating 
one of said processors to said fife one of said 
processors to request that said first auxiliary pro- 
cess read a portion of sard object code file; 
reading said portion of said object code file by said 
first auxiliary process; and 
communicating a second packet containing the 
read portion from said file one of said processors 
to said initiating one of said processors. 

3. The method of claim 2 wherein one of said 
processors is designated a host processor for ex- 
ecuting a function for managing processor assign- 
ment and said allocating step is further 
CHARACTERIZED BY 

communicating a third packet to said host one of 
processors to request assignment of one of said 
processors executing said new program; 
assigning one of said processors for executing said 
new program by the managing function; and 



communicating the assignment via a fourth packet 
to said initiating one of processors from said host 
one of said processors. 

4. The method of claim 3 
5 CHARACTEFUZED IN THAT 

said portion of said object code til contains a first 
set of processor assignment parameters and said 
initiating processor has a second set of processor 
assignment parameters stored in a subset of said 

10 first primary process's process information and 
said tiiird packet contains said first and s cond 
sets of processor assignment parameters and said 
assigning step comprises the steps of reading said 
sets of processor assignment parameters from said 

75 third packet and 

designating said executing one of said processor 
for the processor assignment in response to said 
sets of processor assignment parameters. 

5. The method of claim 3 
20 CHARACTERIZED IN THAT 

said process Infonmation transforms said first pri- 
mary process to function as a primary process and 
said transferring step comprises the steps of read- 
ing said process information by said initiating one 

25 of said processors; 

forming said process information into a fiftii packet 
by said initiating one of s^d processors; and 
transmitting said fiftti packet from said initiating on 
of said processors to said executing one of said 

30 processors. 

6. The method of claim 5 
CHARACTERIZED IN THAT 

said transforming step comprises steps of reading 
said process Information from said flfth packet; 
35 storing said process information in said second 
auxiliary process; and 

converting said second auxiliary process into a 
second primary process. 

7. Apparatus in a multiprocessor system for 
40 initiating the execution of a new program repre- 
sented in a object code file and said multiproces- 
sor system having a plurality of processors, 
CHARACTERIZED BY 

means for determining a file one of said processors 
45 associated with said object code file of said new 
program by a first prinyary process being executed 
by an initiating one of said processors; 
means for creating a first auxiliary process on said 
file one of said processors upon said file one of 
50 said processors having been determined to be 
different from said initiating one of said processors; 
means for allocating an exeucting one of said pro- 
cessors to execute said object code file of said 
new program; 

55 means for creating a second auxiliary process on 
said executing one of said processors upon said 
executing one of said processors being different 
from said initiating one of said processors; 
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means for transferring process information from 
said first primary process to said second auxiliary 
process; 

means for transforming said second auxiliary pro^- 
cess into a second primary process; and 
means for executing said object code file by said 
second primary process In conjunction with said 
first auxiliary process. 

8. The apparatus of claim 7 
CHARACTERIZED IN THAT 
said allocating means comprises: 

means for communicating a first packet from said 
initiating one of said processors to said file one of 
said processors to request that said first auxiliary 
process read a portion of said object code file; 
means for reading said portion of said object code 
file by said first auxiliary process; and 
means for communicating a second packet contain- 
ing the read portion from said file one of said 
processors to said Initiating one of said processors. 

9. The apparatus of claim 8 wherein said ap- 
paratus has designated a host one of said proces- 
sors and said host one of said processors execut- 
ing a function for managing processor assignment 
and said allocating means further 
CHARACTERIZED BY 

means for communicating a third packet to said 
host one of said processors to request processor 
assignment for said new program; 
means for assigning one of said processors for 
executing said new program by the managing func- 
tion; and 

means for communicating the assignment via a 
fourth packet to said initiating one of said proces- 
sors from said host one of said processors. 

10, The apparatus of claim 9 
CHARACTERIZED IN THAT 

said portion of said object code file contains a first 
set of processor assignment parameters and said 
initiating processor has a second set of processor 
assignment parameters stored in a subset of said 
first primary process's process information and 
said third packet contains said first and second 
sets of processor assignment parameters and said 
assigning means comprises means for reading said 
sets of processor assignment parameters from said 
third packet; and 

means for designating said executing one of said 
processor for the processor assignment in re- 
sponse to said sets of processor assignment pa- 
rameters. 

11, The apparatus of claim 9 
CHARACTERIZED IN THAT 

said process information transforms said first pri- 
mary process to function as a primary process and 
said transferring means comprises means for read- 
ing said process information by said initiating one 
of said processors: 



means for forming said process information into a 
fifth packet by said initiating one of said proces- 
sors; and 

means for transmitting said fifth packet from said 
5 initiating on of said processors to said executing 

one of said processors. 

12. The apparatus of claim 1 1 

CHARACTERIZED IN THAT 

said ti^sforming means comprises means for 
10 reading said process information from said fifth 

packet; 

means for storing said process information in said 
second auxiliary process: and 
means for converting sajd second auxiljary process 
75 into a second primary process. 
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